On the connection between the conjugate gradient method and quasi-Newton methods on quadratic problems

It is well known that the conjugate gradient method and a quasi-Newton method, using any well-defined update matrix from the one-parameter Broyden family of updates, produce the same iterates on a quadratic problem with positive-definite Hessian. This equivalence does not hold for any quasi-Newton method. We discuss more precisely the conditions on the update matrix that give rise to this behavior, and show that the crucial fact is that the components of each update matrix is choosen in the last two dimensions of the Krylov subspaces defined by the conjugate gradient method. In the framework based on a sufficient condition to obtain mutually conjugate search directions, we show that the one-parameter Broyden family is complete. We also show that the update matrices from the one-parameter Broyden family is almost always well-defined on a quadratic problem with positivedefinite Hessian. The only exception is when the symmetric rank-one update is used and the unit steplength is taken in the same iteration, in this case it is the Broyden parameter that becomes undefined.